Enforce in bootstrap that doc must have stage at least 1 #145011

Kobzol · 2025-08-06T16:34:53Z

Following with the bootstrap cleanups, this time around doc steps. Should be pretty straightforward, because the supporting infrastructure was already there.

The only thing I found a bit fishy is using Mode::ToolBootstrap as a "catch-all" mode for non-rustc-private steps in tool_doc!, but I don't think that we need to distinguish the tools in some special way when documenting them, apart from supporting rustc_private.

Before, x doc more or less defaulted to what we call stage 2 now. Now it is properly stage 1, so e.g. x doc compiler documents the compiler using the stage0/beta rust(do)c.

r? @jieyouxu

try-job: dist-aarch64-msvc

bors · 2025-08-07T03:56:53Z

☔ The latest upstream changes (presumably #145020) made this pull request unmergeable. Please resolve the merge conflicts.

rustbot · 2025-08-08T06:18:01Z

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

Kobzol · 2025-08-08T06:19:39Z

Rebased over the two merged PRs, should be ready now.

@rustbot ready

jieyouxu

Thanks for working on this! The changes look good in the general sense that it does feel more self-consistent. I do have some questions though that occurred to me when looking at the changes.

src/bootstrap/src/core/config/config.rs

src/bootstrap/src/core/build_steps/doc.rs

jieyouxu · 2025-08-10T03:57:45Z

src/bootstrap/src/core/builder/tests.rs

    #[test]
+    #[should_panic]
    fn doc_compiler_stage_0() {
        let ctx = TestCtx::new();
-        insta::assert_snapshot!(
-            ctx.config("doc")
-                .path("compiler")
-                .stage(0)
-                .render_steps(), @r"
-        [build] rustdoc 0 <host>
-        [build] llvm <host>
-        [doc] rustc 0 <host>
-        ");
+        ctx.config("doc").path("compiler").stage(0).run();


Question: hm, would we regress any actual use cases of documenting the stage 0 compiler?

We can't document the stage 0 compiler, because we don't have its source code. My mental model around this looks like this:

Every stage has two components attached to it: sources and artifacts.

We do not have the source of stage 0 available.

When we do check/clippy/build/doc, we always work with sources, not artifacts, so it makes no sense for it to work on stage 0.

When we do test, we work with artifacts, so it makes sense (in niche scenarios) to also run it on stage 0.

Right, that makes sense 👍. I.e. source-availability vs artifact-availability.

src/bootstrap/src/core/build_steps/doc.rs

jieyouxu · 2025-08-10T04:08:20Z

src/bootstrap/src/core/build_steps/doc.rs

+                    // Build rustc docs so that we generate relative links.
+                    run.builder.ensure(Rustc::from_build_compiler(run.builder, compilers.build_compiler(), target));


Remark: hah, this is not super obvious...

src/bootstrap/src/core/builder/tests.rs

src/bootstrap/src/core/build_steps/dist.rs

jieyouxu · 2025-08-10T04:24:48Z

src/bootstrap/src/core/builder/tests.rs

+        [doc] rustc 2 <host> -> std 2 <host> crates=[]
+        [doc] rustc 2 <host> -> std 2 <target1> crates=[]


Remark: hm, this is always a bit funny, when we run a std doc step with

crates=[alloc,compiler_builtins,core,panic_abort,panic_unwind,proc_macro,rustc-std-workspace-core,std,std_detect,sysroot,test,unwind]

but then again later with

crates=[]

then I think we rely on the fact that they can unify to not have this invalidate the previous build cache (?)

Yes, I think so. I did some experiments with x clippy compiler steps with and without unification and it looked like the cache is not being broken, but we'll see I guess.

jieyouxu · 2025-08-10T04:35:17Z

src/bootstrap/src/core/builder/tests.rs

+        [doc] book (book) <host>
+        [doc] book/first-edition (book) <host>
+        [doc] book/second-edition (book) <host>
+        [doc] book/2018-edition (book) <host>
+        [build] rustdoc 1 <host>
+        [doc] rustc 1 <host> -> standalone 2 <host>


Question: wait, are we building less than we should be? I.e. there are two hosts, $\text{host}_{1} := \text{"host"}$ and $\text{host}_{1} := \text{"TEST-TRIPLE-1"}$. Or is this an instance where the "host" is actually the build platform and not actually the host platform? I.e. under the scenario where bootstrap is ran on a different platform than where the stage 1 compiler would be ran (?)

Hm... or taking a step back, what does the "hosts" config actually mean in bootstrap?

That's a very good question 😆 When you specify hosts, you will essentially override the default host build target, see this:

rust/src/bootstrap/src/core/config/config.rs

Line 859 in d8dca72

config.hosts = if let Some(TargetSelectionList(arg_host)) = flags_host {

In terms of what hosts means, the documentation is here:

rust/src/bootstrap/src/lib.rs

Line 218 in 213d946

hosts: Vec<TargetSelection>,

. hosts determines for what targets we should build host tools, while targets determines for what targets we should build the stdlib.

But yeah, it's super confusing.

Argh, I think I asked you about this once already, but I keep forgetting the actual distinction.

jieyouxu · 2025-08-10T04:40:54Z

@rustbot author

Kobzol · 2025-08-10T09:29:57Z

@rustbot ready

As they were previously.

bors · 2025-08-10T16:56:33Z

📌 Commit 8b4d941 has been approved by jieyouxu

It is now in the queue for this repository.

bors · 2025-08-11T06:00:55Z

⌛ Testing commit 8b4d941 with merge a6620a4...

bors · 2025-08-11T09:02:11Z

☀️ Test successful - checks-actions
Approved by: jieyouxu
Pushing a6620a4 to master...

github-actions · 2025-08-11T09:05:04Z

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 53af067 (parent) -> a6620a4 (this PR)

Test differences

Show 7 test diffs

Stage 0

core::builder::tests::snapshot::build_compiler_lld_opt_in: [missing] -> pass (J0)
core::builder::tests::snapshot::doc_reference: [missing] -> pass (J0)
core::builder::tests::snapshot::test_lld_opt_in: pass -> [missing] (J0)

Additionally, 4 doctest diffs were found. These are ignored, as they are noisy.

Job group index

J0: aarch64-apple, aarch64-gnu, aarch64-gnu-llvm-19-2, aarch64-msvc-2, i686-gnu-2, i686-gnu-nopt-2, i686-msvc-2, x86_64-apple-1, x86_64-gnu, x86_64-gnu-llvm-19-1, x86_64-gnu-llvm-19-3, x86_64-gnu-llvm-20-1, x86_64-gnu-llvm-20-3, x86_64-gnu-nopt, x86_64-gnu-stable, x86_64-mingw-2, x86_64-msvc-2

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard a6620a45bd29575cce67b6a0ab2956aef105e324 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

pr-check-2: 2651.2s -> 4054.2s (52.9%)
dist-apple-various: 4077.2s -> 6062.0s (48.7%)
dist-aarch64-linux: 5532.0s -> 8017.3s (44.9%)
x86_64-apple-2: 5299.8s -> 3660.4s (-30.9%)
aarch64-apple: 8935.9s -> 6582.2s (-26.3%)
x86_64-apple-1: 9952.6s -> 8142.2s (-18.2%)
pr-check-1: 1702.0s -> 1448.2s (-14.9%)
aarch64-gnu-llvm-19-2: 2511.1s -> 2200.1s (-12.4%)
i686-gnu-1: 8311.9s -> 7430.5s (-10.6%)
x86_64-gnu-llvm-19: 2756.6s -> 2468.1s (-10.5%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

rust-timer · 2025-08-11T10:32:54Z

Finished benchmarking commit (a6620a4): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (secondary -3.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.4%	[-3.6%, -3.1%]	2
All ❌✅ (primary)	-	-	0

Cycles

Results (secondary 3.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.3%	[2.8%, 4.0%]	4
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 463.403s -> 464.586s (0.26%)
Artifact size: 377.34 MiB -> 377.31 MiB (-0.01%)

Enable RUST_BACKTRACE=1 on CI We should really see the backtrace if something in bootstrap fails on CI. #145011 (comment) doesn't show many details.

Kobzol · 2025-08-11T14:14:29Z

Hmm, the pr-check-2 regression looks genuine, I'll investigate.

Kobzol · 2025-08-11T14:22:29Z

#145253 should fix it.

lolbinarycat · 2025-08-11T18:32:57Z

e.g. x doc compiler documents the compiler using the stage0/beta rust(do)c.

So the previous meaning of stage1 (build rustdoc with current source and beta compiler) has been erased?

This would be a pretty big change, eliminating certain workflows but also meaning the cfg(bootstrap) will never need to be used in rustdoc, and making synchronized changes between rustdoc and libtest will be much easier.

cc @GuillaumeGomez Does this unblock any of your PRs?

Kobzol · 2025-08-11T18:36:58Z

So the previous meaning of stage1 (build rustdoc with current source and beta compiler) has been erased?

Could you please clarify what do you mean by "previous meaning of stage1"?

This PR shouldn't really make any large changes to rustdoc workflow. It just makes staging of doc steps consistent with the rest of bootstrap; after the stage 0 redesign, most steps were inconsistent and still living in pre-stage0-redesign world.

GuillaumeGomez · 2025-08-11T19:01:29Z

I think @lolbinarycat is talking about the libtest blockers on my PRs. And no, I think it's unrelated.

lolbinarycat · 2025-08-11T19:19:24Z

right, because x doc --stage 1 and x test --stage 1 don't actually use the same rustdoc... hm.

Kobzol · 2025-08-11T20:11:41Z

I still have to get to fixing test steps, so far I mostly fixed build, check, tools, doc, codegen backends and now working on Clippy. Then I'll work on dist and test and it should be mostly done.

…ieyouxu Document compiler and stdlib in stage1 in `pr-check-2` CI job This restores the original behavior pre-rust-lang#145011 (I thought that stage 2 makes more sense here, but it made the job ~30m slower, which is bad). Let's see what will be the "new" duration, it should be ~55 minutes. r? `@jieyouxu`

…ieyouxu Document compiler and stdlib in stage1 in `pr-check-2` CI job This restores the original behavior pre-rust-lang#145011 (I thought that stage 2 makes more sense here, but it made the job ~30m slower, which is bad). Let's see what will be the "new" duration, it should be ~55 minutes. r? ``@jieyouxu``

…ieyouxu Document compiler and stdlib in stage1 in `pr-check-2` CI job This restores the original behavior pre-rust-lang#145011 (I thought that stage 2 makes more sense here, but it made the job ~30m slower, which is bad). Let's see what will be the "new" duration, it should be ~55 minutes. r? ```@jieyouxu```

…ieyouxu Document compiler and stdlib in stage1 in `pr-check-2` CI job This restores the original behavior pre-rust-lang#145011 (I thought that stage 2 makes more sense here, but it made the job ~30m slower, which is bad). Let's see what will be the "new" duration, it should be ~55 minutes. r? ````@jieyouxu````

…ieyouxu Document compiler and stdlib in stage1 in `pr-check-2` CI job This restores the original behavior pre-rust-lang#145011 (I thought that stage 2 makes more sense here, but it made the job ~30m slower, which is bad). Let's see what will be the "new" duration, it should be ~55 minutes. r? `````@jieyouxu`````

…ieyouxu Document compiler and stdlib in stage1 in `pr-check-2` CI job This restores the original behavior pre-rust-lang#145011 (I thought that stage 2 makes more sense here, but it made the job ~30m slower, which is bad). Let's see what will be the "new" duration, it should be ~55 minutes. r? ``````@jieyouxu``````

rustbot assigned jieyouxu Aug 6, 2025

Kobzol force-pushed the bootstrap-doc branch from ad1b0bc to a3b3576 Compare August 8, 2025 06:15

Kobzol marked this pull request as ready for review August 8, 2025 06:17

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 8, 2025

jieyouxu reviewed Aug 10, 2025

View reviewed changes

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 10, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Aug 10, 2025

Kobzol added 10 commits August 10, 2025 11:39

Forbid documenting anything on stage 0

b3d90af

Fix staging for doc compiler

980fe06

Update Reference doc step

02fc091

Update RustcBook doc step

5217edd

Update Standalone and Releases doc steps

c35e847

Fix documentation of tools

bec8e1d

Update Std doc step

8f3f060

Update doc CI steps stage 2

fe01f58

As they were previously.

Update tests

6443dc0

Add change tracker entry

c843e1d

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 10, 2025

bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 11, 2025

bors merged commit a6620a4 into rust-lang:master Aug 11, 2025
12 checks passed

rustbot added this to the 1.91.0 milestone Aug 11, 2025

Kobzol deleted the bootstrap-doc branch August 11, 2025 09:39

This was referenced Aug 11, 2025

docs: autogenerate compiler flag stubs based on -Zhelp #142135

Open

install docs for each target in different directory #142478

Open

Add relnotes-api-list in-tree tool #143053

Draft

Enforce in bootstrap that clippy must have stage at least 1 #145131

Open

bors added a commit that referenced this pull request Aug 11, 2025

Auto merge of #145215 - Kobzol:rust-backtrace, r=marcoieni

fce0e74

Enable RUST_BACKTRACE=1 on CI We should really see the backtrace if something in bootstrap fails on CI. #145011 (comment) doesn't show many details.

Kobzol mentioned this pull request Aug 11, 2025

Document compiler and stdlib in stage1 in pr-check-2 CI job #145253

Open

		// Build rustc docs so that we generate relative links.
		run.builder.ensure(Rustc::from_build_compiler(run.builder, compilers.build_compiler(), target));

		[doc] rustc 2 <host> -> std 2 <host> crates=[]
		[doc] rustc 2 <host> -> std 2 <target1> crates=[]

Enforce in bootstrap that doc must have stage at least 1 #145011

Enforce in bootstrap that doc must have stage at least 1 #145011

Uh oh!

Conversation

Kobzol commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bors commented Aug 7, 2025

Uh oh!

rustbot commented Aug 8, 2025

Uh oh!

Kobzol commented Aug 8, 2025

Uh oh!

jieyouxu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jieyouxu Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jieyouxu commented Aug 10, 2025

Uh oh!

Kobzol commented Aug 10, 2025

Uh oh!

bors commented Aug 10, 2025

Uh oh!

bors commented Aug 11, 2025

Uh oh!

bors commented Aug 11, 2025

Uh oh!

Uh oh!

github-actions bot commented Aug 11, 2025

Test differences

Stage 0

Job duration changes

Uh oh!

rust-timer commented Aug 11, 2025

Overall result: no relevant changes - no action needed

Uh oh!

Kobzol commented Aug 11, 2025

Uh oh!

Kobzol commented Aug 11, 2025

Uh oh!

lolbinarycat commented Aug 11, 2025

Uh oh!

Kobzol commented Aug 11, 2025

Uh oh!

GuillaumeGomez commented Aug 11, 2025

Uh oh!

lolbinarycat commented Aug 11, 2025

Uh oh!

Kobzol commented Aug 11, 2025

Uh oh!

Uh oh!

Kobzol commented Aug 6, 2025 •

edited

Loading

jieyouxu Aug 10, 2025 •

edited

Loading